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ALTERNATING PROJECTIONS ON NON-TANGENTIAL 

MANIFOLDS. 

FREDRIK ANDERSSON AND MARCUS CARLSSON 

o 

^vj , Abstract. We consider sequences {B^.)'^_„ of points obtained by project- 

ing back and forth between two manifolds Mi and AI2, and give conditions 
r^ ' guaranteeing that the sequence converge to a hniit Boo G Mi r\M2- Our mo- 

tivation is the study of algorithms based on finding the limit of such sequences, 
which have proven useful in a number of areas. The intersection is typically a 
set with desirable properties, but for which there is no efficient method of find- 
ing the closest point Bopt in Mi n M2- We prove not only that the sequence 
^^ ' of alternating projections converges, but that the limit point is fairly close to 

, J* , Bopt, in a manner relative to the distance ||i3o — ^opt ||, thereby significantly 

^ /^ l ■ improving earlier results in the field. A concrete example with applications to 

' ' frequency estimation of signals is also presented. 

1. Introduction 

, ^ Let /C be a finite dimensional Hilbert space over M and let A^i, A^2 C /C be 

.1^ . manifolds. Suppose that for any B <E K, the closest point on Mj, j — 1,2, is 

If-y ' well defined and lets denote it by ■Kj{B). Let the corresponding projection onto 

C^ • the intersection A^i n A^2 be denoted by tt{B). Suppose that we are interested 

^^ ' in finding the closest point on A^i n A^2 and that the "projection operators" tti 

r — . and 7r2 can be efficiently computed, whereas tt can not. The issue treated here 

f^ ' is how to use tti and 7r2 to obtain an approximation of n. A classical result by 

von Neumann [18] says that ii A4i and M2 are affine linear manifolds, then the 

sequence of alternating projections 

(1.1) 7r,{B), 7r2(^i(i?)), 7ri(^2(7ri(i?))), ^2(7ri(^2(7ri(i?)))), . . . 

. , converges to tt{B). Moreover, the convergence rate is determined by the angle 

j^ ■ between A^i and A^2- This paper is concerned with extensions of this result to 

non-linear manifolds. 

Hence, given B E K,, let Bi = tti{B) and 

7ri(Sfc) k, is even, 
T^2{Bk) k, is odd. 

In contrast to the case where AAj is an affine linear manifold, B^o 7^ ''^{B). However, 
given that AAj behave nicely, we may expect Boo ~ '^{B)- 

Alternating projection schemes of this kind have been used in a number of ap- 
plications, cf. [7, 10, 12, 14, 15, 17, 16, 19]. For instance, /C can be the set M^ „ 
of 777, X n-matrices, and the manifolds Mj are subsets with a certain structure, e.g. 
matrices with a certain rank, self-adjoint matrices, Hankel or Toeplitz matrices 
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etc. Alternating projection schemes between several linear subsets in the infinite 
dimensional setting was recently investigated in [2]. Much emphasis has been put 
towards the usage of alternating projections for the case of complex manifolds, see 
for instance [3, 4]. Connections to the EM algorithm are given in [5]. 

In [7] , Zangwill's Global Convergence Theorem [20] is used to motivate the con- 
vergence of the alternating projection scheme above. From Zangwill's theorem it 
is possible to deduce that if the sequence {Bk)^i is bounded and the distance to 
All n AI2 is strictly decreasing, then (-Bfe)^i has a convergent subsequence to a 
point Boo G -Ml n 7M2, i-e., a result related to the fact that any bounded sequence 
in a compact set has a convergent subsequence. Thus, the use of Zangwill's theorem 
in this context does not provide any information about whether the limit point Bqo 
exists, or if so, whether it is close to tt{B). 

Recently, A. Lewis and J. Malick presented stronger results, valid under more 
restrictive conditions on Ali and A^2- Before discussing their results in more detail, 
we give two simple examples that illustrate some of the difficulties that may arise 
when using alternating projections on non-linear manifolds. 

Example 1.1. Let IC = M? and set Mi = {{t,{t + 1)(3 - t)/4) : t e M} and 
M2 = K X {0}. It is easily seen that 7ri((l,0)) = (1,1) and 7r2((l,l)) = (1,0), and 
hence the sequence of alternating projections does not converge, cf. Figure 1 . On 
the other hand, if (1 -|- e, 0) G AI2, e > 0, is used as a starting point, the sequence 
of alternating projections will converge to (3, 0) G Ali n Al2- 




Figure 1. An example demonstrating that the algorithm can 
get stuck in loops where even no subsequence converges to a point 
in the intersection. 



In general, it seems reasonable to assume that if the starting point is sufficiently 
close to an intersection point, then the sequence does converge to a point in the in- 
tersection. The next example shows that this is not the case of limited smoothness. 

Example 1.2. Without going in to the details of the construction, we note that one 
can construct a C^-function / such that, with A^i = R x {0} and A^2 — {{t, f{t)) : 
t G M}, the sequence of alternating projections can get stuck in projecting back and 
forth between the same two points. Figure 2 explains the idea. 

However, if we have / e C^ and /'(O) 7^ 0, it is hard to imagine how to make 
a similar construction work. This is indeed impossible if we assume additional 
smoothness, which follows both from the present paper and [13]. In the terminology 
of the latter, the condition /'(O) ^ implies that Mi and M2 are transversal at 
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Figure 2. Alternating projections stuck in a loop. 

(0, 0). In general, given C^-manifolds Mi, M2 and a point A e Mi n AI2, we say 
that A is transversal if 

(1.3) TmAA)+TmAA)=IC, 

where Tm . (A) denotes the tangent-space of Mj at A, j = 1,2. The main result in 
[13] is roughly the following: 

Theorem 1.3. Let Mi and M2 be C^ -manifolds and let A E Mir\M2 be transver- 
sal. If B is close enough to A, then the sequence of alternating projections (i?fe)fcJLi 
given by (1.2) converges to a point B^o in Mi f] M2- Moreover, 

\\B^-tt{B)\\<2\\A~B\\. 

The improvement over Zangwill's theorem is thus that the entire sequence con- 
verges, and that the assumption of boundedness no longer is necessary. Moreover, 
the limit B^o is not too far off from 7r(i?), (although in relative terms, i.e. com- 
paring with 6\st{B,Mi n M2) = \\B — 7r(i?)||, it need not be particularly close 
either). 

On the other hand, the assumption of transversalsality is rather restrictive. To 
demonstrate the essence of the transversality assumption, we now present two cases 
which are not covered by the above result, but where the conclusion still holds. 




(0,0) 



Figure 3. Lack of transversality due to tangential curves. 



Example 1.4. With Mi 



X {0} and M2 = {{t,t^) ■ t G M}, transversahty is 



not satisfied at A — (0,0) (since Tmi{A) ~ Tm2{^) ~ ^1); but it is not hard to 
see that the sequence of alternating projections still converges to (0, 0), (see Figure 
3). 

The difference between Example 1.2 and 1.4 is that in the latter case the mani- 
folds are more regular. 

Example 1.5. With /C = R^ and TWi == M x {0}^ and M2 = {{t,t,t'^) : t e K}, 
transversality is not satisfied at A = (0,0), but again it seems plausible that the 
sequence of alternating projections converges to (0, 0). See Figure 4. 
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Figure 4. Lack of transversality due to too low dimensionality. 



In Example 1.5, the two manifolds clearly sit at a positive angle, but this situation 
is not covered by Theorem 1.3, since the manifolds are of too low dimension to 
satisfy the transversality assumption. In fact, if /C has dimension n and Aij has 



dimension to. 



J 



1,2, the transversality (1.3) can never be satisfied if mi 



1712 < n. But for many applications of practical interest, one has mi + ?7i2 « n. 
We will introduce a concept which we call non-tangential, which loosely speaking 
says that the manifolds should have a positive angle in directions perpendicular to 
A^i n Al2- The point (0,0) in Example 1.5 is non-tangential, whereas the same 
point in Example 1.4 is not. A proper definition of non-tangentiality is given in 
Definition 4.3. A simplified version of our main result is given in what follows. 

Theorem 1.6. Given a non-tangential point A G A^i n AI2 there exists an s > 
such that the sequence of alternating projections (1.2) converges to a point Boo G 
A^i n A^2 7 given that ||_B — Aj| < s. Moreover, given any e > one can take s such 
that 

\\B^-7:{B)\\ <e\\B-7r{B)\\. 

The full version of the main theorem is given in Section 6. 

The improvement over Theorem 1.3 mainly consists of two items. Primarily, 
the assumption that the surfaces be non-tangential is not at all restrictive, and 
in particular there is no implication on the dimensions of A^i and A^2- For the 
applications we are aware of, the set of tangential points is very small, if it exists at 
all. Secondly, as has been highlighted before, we are usually interested not just in 
any point of A^i n A^2, but the closest point tt{B). Here the theorem says that in 
relative terms, i.e. after dividing with the distance to A^i n AI2, the error is small if 
the distance to Mi n A^2 is small. It is also worth mentioning that we only assume 
that the manifolds are C^ , although the manifolds are C°° in the applications that 
we are aware of. 
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In order to motivate the need for certain technicalities in the abstract setting, 
as well as the need for a theorem concerning low-dimensional manifolds, we will 
develop the theory in parallel with an example, namely that when /C is the space 
M„^„(C) of complex n x n-matrices, A^i is the set of Hankel matrices and 7M2 
the set of matrices of rank at most k. The former is a subspace of real dimension 
2(n — 1), whereas the latter is not actually a manifold. However, it is "locally" a 
manifold of real dimension 2(2nk — k^) at all matrices A with rank precisely k, 
(which clearly constitutes the majority of matrices in the set). If k is small and 
A is a non-tangential intersection point, then dim(Tv)j) -I- dim(TA/(2) = 2{2nk — 
k'^ + n — 1), which is much less than dim/C = 2n^, unless k is close to n. Another 
thing that does not match between this example and the theory outlined above is 
that the projection 7r2 is not uniquely defined at certain points. This is common 
in algorithmic theory and can be dealt with by using point-to-set maps, following 
Zangwill [20]. However, in our case, this is not necessary since we show that the 
projections are locally well defined near non-tangential points, (Proposition 3.3). 
Moreover, from a practical perspective this is unnecessary since such points are 
rarely encountered in applications. 

For the above example, the interest in A^i n 7M2 lies in the fact that such 
sequences are samplings of functions which are sums of k exponential functions. 
Given a function / on an interval the alternating projections algorithm can then 
be used to find approximations of / by sums of k exponential functions, which is 
a problem of great practical interest. We demonstrate the idea in Figure 5. More 
thorough examples are conducted in [1]. 

2. Case study; rank k matrices versus Hankel matrices 

We include this section for the reader to get the picture of a typical application. 
However, this particular application is treated in detail in [1], and therefore we will 
be very brief here. In the setting considered by Lewis and Malick, the fact that 
A4i n A^2 is itself a manifold (around a point A, say) follows by the tranversality 
assumption (1.3) and standard differential geometry. In our (more general) setting, 
A^i n 7W2 may fail to be a manifold. However, in the example developed in this 
section, A^i n 7W2 will be a manifold locally. In fact, it is not hard to see that it 
will always happen for algebraic manifolds, which is the case for all of the various 
applications presented in [7]. 
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Figure 5. Signal with five exponentials. Left panel: original sig- 
nal (black solid line); signal with white noise (gray circles); and 
the reconstruction using the alternating Hankel projection method 
(black circes). Right panel: original exponential nodes (black 
dots), estimated nodes from noisy signal (gray circles.) 



Let n G N be fixed, let /C = M„.„(C) and let "H C /C be the set of Hankel 
matrices, e.g., matrices of the form 

/ fli 02 as ■■■ a„ ^ 



(2.1) 



02 
03 



V o„ 



03 



O^ 



Ori+1 



02n-2 



On+1 



02n-2 
02n-l / 



"H is a linear subspace and, hence, a manifold at each point, of (real) dimension 
2(2n — 1). Denoting the matrix in (2.1) by H{a), where a = (ai, . . . ,a2n-i), we 
obtain a natural chart for T-L, (by identifying C with R^ in the obvious way). 

Given k < n, we denote by TZk C /C the set of matrices of rank less than or equal 
to k. We wish to do alternating projections as outlined in the introduction between 
"H and TZk, which is slightly complicated by the fact that TZk is not a manifold. 
However, it turns out that TZk is locally a manifold of (real) dimension 2(2nk — fc^), 
apart from at some exceptional points. More precisely, suppose A € TZk, and use 
the singular value decomposition of A to find a a G (K^)'^ and unitary matrices 
Ua-,Va e M„^fc such that 



/ cfa: 



(2.2) 



A = Va 











crA,2 



\ 



V 



u\ 



VAlaU; 



: •. •. 

• • • aA^k I 

The vast majority of matrices in TZk satisfy 

(2.3) OAA > <yA.2 > ■■■ > (TA,k > 0, 

and an arbitrarily small numerical perturbation will yield distinct singular values. 
The subset oi TZk satisfying (2.3) will be denoted TZf, where d stands for distinct. 
If A^ is a manifold and its closure A4 is such that TW \ Al is a union of manifolds 
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of lower dimension than Ai , we will say that ^A \ ^A is thin. The proof of the 
following proposition can be found in [1]. 

Proposition 2.1. T^-J? is a manifold of (real) dimension 2{2nk — fc^). Moreover, 
TZk = nf and TZk \ TZf is thm. 

The structure of the set TZk around the exceptional points in TZkYR-f can be rather 
complicated, and although the sequence generated by the alternating projection 
scheme could theoretically approach such a point, we have not done any analysis of 
convergence properties in this setting. This seems to be a hard problem. The rate 
of convergence is however very poor around such points (compare Figure 1 with 
Figure 3). The first contains 12 iterations and the second 100. 

We note that by the Eckart- Young theorem [11], it is easy to project any matrix 
B € M[„^„ onto its closest point in TZk while using the Frobenius norm 



\B\\ 



Specifically, the theorem says that HE — VbIoUb is a singular value decomposition 
of B, then the closest point in TZk is given by replacing I^ with It where r — 
(cTi, . . . , (Tfe, 0, . . . , 0). The closest point is thus unique as long as the singular values 
are distinct, which is always the case when working with "real numerical" data, so 
we will for simplicity treat the projection onto TZk as a well defined map which we 
denote by tttj^. . More stringently one could work with "point to set"-maps, as in [7] 
and [20]. 

The last manifold that needs to be discussed is 




i.e., the set of Hankel matrices with rank < k. 
a G C, the matrix 



/ 1 



a 



a 



It is easily seen that, given any 



a 



(2.4) 



H{a) = 



defines a rank 1 Hankel matrix, and thus 



a 



i,2n-3 



a 



„2n-3 



(2.5) 



S){c,a) 






CjH{aj 



has rank k. This is the typical case, as we will show below. Let us denote the image 
of S) intersected with TZ'l by "H^, where n stands for "nice". Again, the analysis 
is complicated by some exceptional points. For example, -j^H(a) is a Hankel 
matrix of rank j which is not covered by i^. Moreover, rank k Hankel matrices 
containing summands of this type are usually inside TZf,, but despite that, some 
investigations show that the manifold structure of Tik collapses around such points. 
The sup-script "n"(= nice) is thus really more restrictive than "rf"(= distinct). 
Nevertheless, Tik\Tik is thin and never encountered in practice, so we omit a study 
of such cases. The proof of the following result can be found in ]1]. 
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Proposition 2.2. 

% is a 2{2n — 1) -dimensional linear subspace. 

WJ. is a Ak- dimensional manifold which is dense in Tik, its complement is 

thin. 

The map ttt^^ is well defined on all points in M„.„ except a for thin subset. 

We now explain why one would like to do alternating projections between "H and 
TZk. Let TT-H be the orthogonal projection onto H, and let tt{B) denote the closest 
point to B in "H n TZk- Say we have a sampled function on an interval, e.g., the 
signal in Figure 5. Denote the corresponding sequence by / and note that 

2n-l 

(2.6) El/^l'("-|^'-H) = l|i^(/)ll2- 

We denote the weight sequence in (2.6) by w and the corresponding norm on C^"^^ 
by 11/11^. By Proposition 2.2, the function g defining tt{B) — H{g) is almost surely 
in the range of Jo, i.e. of the form 

k 

El I \1n-1 

i=i 

Such functions are precisely what one encounters when sampling sums of k expo- 
nential functions, and hence g is the closest such function to / in the norm given 
by (2.6). Approximating a given function with sums of a predetermined number of 
exponentials is a problem with a large number of applications. There is, however, 
no computationally efficient method for computing g. In contrast, the projections 
TT-ji^, and TTfi are relatively easy to compute, and hence, by the results of this paper, 
we can get a fairly good approximation of g by finding the limit of the sequence of 
alternating projections, starting at H{f). 

For the sake of efficient computations, care has to be taken concerning the im- 
plementation of 'n-Rk ^^d TT-H . Constructing fast algorithms for this purpose is one 
of the topics of a companion paper [1] , where we also discuss other aspects of this 
specific application of alternating projections as well as give proofs of the above 
claims. In the present paper, we will continue discussing this application in Section 
7. 



3. Preliminaries 

Let /C be a Hilbert space of dimension n e N. Given A e /C and r > we 
write B{A,r) or BiciA^r) for the open ball centered at A with radius r. Since /C is 
finite-dimensional it has a unique Euclidean topology. Any subset M oi K, will be 
given the induced topology from /C. 

Definition 3.1. We say that Ai d IC is locally an m- dimensional C^ -manifold 
around A (z Ai if there exists ri > and a C^-map (p : ,8R™(0,ri) — > /C with the 
following properties: 

• d(j){x) is injective for all x G yBR™(0,ri), 

• 0(0) = A, 

• (j) is a homeomorphism onto an open neighborhood of A in M. 
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This is in line with the standard definition of C^-manifolds, see Theorem 2.1.2 
[6] for a number of equivalent definitions. As a consequence of the homeomorphism 
condition, note that there exists an si > such that 

(3.1) MnB,c{A,si)^\m(l)nBic{A,si), 

where Imc^ denotes the image of (f>. Given B G Im0, there exists a unique x e B{0, ri) 
such that B — (t>{x). We will without further comment denote this xhy xb- All 
the manifolds considered in this paper are at least C^ , and hence we have 

(3.2) 4>{x) = C + d(j){xc)ix - xc) + o{x - xc) 

where o stands for "little ordo".^ We define the tangent space Tm{B) by Tm{B) = 
Ran d(f>(xB)- It is a standard fact from differential geometry that this definition is 
independent of (j). Moreover, we set 

fMiB) = B + TMiB), 

i.e., Tm{B) is the afSne linear manifold which is tangent to Ai at B. Throughout 
this section, Ai will be a locally C^-manifold at A, where p > 1 and we associate 
with it ri and si as in Definition 3.1 and (3.1). Moreover, there will follow a row of 
decreasing numbers r2, r^ and S2, S3 etc., related to the above A, and the reader 
has to bear in mind where these numbers were defined. The following proposition 
basically says that the afiine tangent-spaces are close to Ai locally. 

Proposition 3.2. Let Ai be a locally C^ -manifold at A. For each 62 > there 
exists S2, < S2 < si, such that for all C G B{A^ S2) C] Ad we have 

{i) dist(B,rvi(C)) <e2||B-C||, B ^ B{A,s2)r\ M. 
(ii) d\st{B,Ai)<e2\\B-C\\, B € ^(^, S2) n Ta^(C). 

Proof. Given r < ri, we first show that ||(/'(a;) — (/)(j/)||/j|.x — y\\ is uniformly bounded 
above and below for x,j/ € yBR™(0,r). By (3.2) and the mean value theorem we 
have 

Uiy) - ^ix)\\ ^ \\d^iz)iy - x)\\ 

for some z on the line between x and y. Now, d(j){z) depends continuously on z 
and its singular values ai(d(j){z)), . . . , am{d(l){z)) depend continuously on the matrix 
entries [9, pl91], hence 

(3.3) 

inf {an{d(f){z))}\\y - x\\ < \\(l){y) - (j){x)\\ < sup {ai{d(j){z))}\\y - x\\. 

By Definition 3.1, am{d(f){x)) is never zero and cZ(i3Rm (0, r)) is compact, so both the 
inf and sup amount to finite positive numbers, as desired, [cl denotes the closure). 
We now prove (i); by (3.2) and the mean value theorem we have 

A\st{B,fM{C)) < \\B - {C + d4>(xc)(xB - xc))\\ ^ \\(I>(xb) - <l>(xc) - d4>(xc)(xB - xc))\\ < 

< supl\\d(f){y) - d(j){xc}\\ : y G BM^(0,r) such that \\y - xc\\ < ||a;B - a;c|| |||a;B - a;c||- 

Since dcj) is continuous on the compact set c/(y8Rm(0,r)), it is also equicontinuous. 
It follows that for each e > we can pick a S > such that 

sup{\\d(f){y) — d(j){x)\\ : x,y e ,BR™(0,r) such that ||j/ — a;||<(5}<e. 



i.e. it stands for a function with the property that o(a;)/||a;|| extends by continuity to and 
takes the value there. 
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Set r2 < min((5/2,r) and let S2, < S2 < si, be such that B e Bic{A,S2) n M 
imphes ||xb|| < r2, which we can do since (f> is a homeomorphisni with (/i(0) = A. 
For B,C G Bic{A, S2), we then have (by equation (3.3)) that 

d\st{B,fMiC)) < e\\xB - xcW < ek\\<j)ixB) - H^c)\\ = ek\\B - C\\, 

where k — {miz^i3g.m(a,r){o'n{d(f){z))})^^ , (cf equation (3.3)). By letting e = e2/k, 
the desired statement follows. 

The proof of (m) is similar. Given C let y be such that B = C + d(f>(xc)y and 
note that <j>{xc + y) E Ai so 

dist(B,M) < Mxc + y) -(C + d4>{xc)y)\\ < sup{|ld0(xc + ty) - d<j)ixc)\\ : i G [0, 1]}| 

Moreover, if S2 is such that ||a;c|| < r, it is easily seen that 

\\y\\<2s2/ sup {<Ji(d(j>(z))}. 

Thus, by picking $2 small enough we can ensure that for all B,C E Bic{A, S2), all 
points of the form xc + ty are bounded by some pregiven 6 > 0. From here the 
proof follows a similar path as {i), we omit the remaining details. D 

The value of £2 will be determined later, so for the moment we will consider it as 
a constant and the corresponding value of S2 will be kept for future use. However, 
r2 was an internal variable in the above proof, and to avoid too many subindices, we 
let r2 take a new value in the proof of the next proposition, (in contrast to the value 
of ri which was given in Definition 3.1 and connected to the fixed point A). The 
next proposition shows that projection on Al is a locally well defined operation. 

Proposition 3.3. Let Ai be a locally C -manifold at A with p > 1. Then there 
exists S3 > and a C^^^ map 

TT:BK{A,s:i)^M 

such that for all B G B]c{A^s^) there exists a unique closest point in A4 which is 
given by 7r(i?). Moreover, C Cz Ai D Bk:{A, S3) equals t:{B) if and only if B — C 1. 
Tm{C). 

Proof. Recall that n is the dimension of K, and m the dimension of M at A. 
By standard differential geometry there exists an r2 < ri and C^^ -functions 
/i, • ■ • , fn-rn ■ ^K" (0, ^2) with the property that 

{TM{(f>{x))) = Span {/i(x), . . . , /„_„(x)} 

for all X e i3Rm(0,r2), (see e.g. Theorem 2.7.7 in [6]). Define a : ,8Rm(0, r2) x 
R"-™ -^ JC via 



71 — m 



<^{x,y) = (j){x) + ^ yift{x). 

Consider the set 5 C /C of points whose multiplicity under a is greater than 1, 
(i.e. all points hit more than once by a). By the inverse function theorem, the set 
(T~^(5) can not have as an accumulation-point, for it says that there exists an 
r < r2 such that g restricted to B^^ (0, r) is a diffeomorphism onto its image. Pick 
ra < min (r, dist(CT~^(5), 0)) and pick S3 such that 

(3.4) M n Bk{A, 2s3) - </)(Sr". (0, r^)) n Bk.{A, 2S3), 
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and 

(3.5) BK{A,S3)cia{BwLA0,r3)). 

Note that, given B e Bjc{A,S3) there exists a unique {xB,yB)s'ach that B = 
(j{{xB,yB)) and moreover ||(a;B, j/_b)|| < rs by (3.5). We define 

t:{B)^^{xb). 

To see that tt is a CP^^-map, let 6* : M" x R""™ -^ M" be given by e{{x,y)) = x 
and note that 

We now show that 7r(i?) have the desired properties. By the construction, 

t:{B) -Be Span {/i(xb), . • . , /„-™(xb)} ± TM{(t>{xB)) = Tm{tt{B)). 

Now suppose C G Al is a closest point to B. Since ||j4 — B|| < S3 we clearly must 
have \\C — A\\ < 2s3 so by (3.4) there exists a xc <E SR™(0,r3) with (/)(xc) — C. 
Since r^ < ri we know that M is completely determined by (j) in the vicinity of 
C. In particular, it makes sense to talk about Tj^(C) and it is easily seen that 
B - C _L Tm{C), for by (3.2) we have 

U{x) -Bf = \\C + d^{xc)ix - xc) + o{x - xc) - B\\ - 
^\\C-Bf + 2(C - B, d^{xc)ix - xc)) + o{\\x - xc\\) 

and hence the scalar product needs to be zero for all x's. Thus there is a y such 
that B — a{{xc,y))- But since (xbtVb) is the unique point with this property, we 
deduce that xb — xc and hence tt{B) — (f){xB) — (f>{xc) = C. This establishes the 
first part of the proposition. Now let C be as in the second part of the proposition. 
As above we have C = 4>{xc) with ||xc|| < r^ and the orthogonality implies that 
there exists a y with B = a{{xc,y)) and again this implies C — 7r(i3), as desired. 

D 

4. NON-TANGENTIALITY 

Suppose now that we are given closed sets A^i and AI2 which locally are mani- 
folds around an intersection point A G A^i n A^2. 

Definition 4.1. An intersection point A will be called regular if there are numbers 
TOi, ?Ti2, rn and p > 1 such that 

• Aij is locally an nij -dimensional C^ -manifold at A, j — 1,2. 

• A^i n AI2 is locally an m-dimensional C^ -manifold at A. 

Note that the set of regular points is clearly a relatively open set in A^i n A^2. 
Next, we introduce angles. For more information on angles, we refer to [13]. 

Definition 4.2. For any regular point A, we define the angle a{A) of Aii and M.2 
at A to be the cos^^ of the number 



{Bi -A,B2- A) 

"^^"''[\\Bi-A\\\\B2-A\\ 



a{A) = lim sup \ -jj^ jj^ ^ : Bj G Mj, \\Bj - A\\ < r and Bj - A J. TM^nM2{A) 
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Given a linear subspace M we let Pm denote the orthogonal projection onto Ai . 
It is easily verified that 
(4.1) 

'^(A) = sup I \\B^ I AUB^'-^AW • ^' ^ ^^^ ^^^ ® TM^nMM) 

= \\PTM^(A)eTMinM2{A)PTM2{A)QTM^nM2(A)\\ ^ II^Tai^ (AJ-Pt^^ (A) ^ -Fr^inAia (A) II • 

If All and M.2 are hyperplanes through the origin, then it is clear that the above 
definition coincides with the classical definition; 

(4.2) COSaclas{A) = (7clas{A) = \\PmiPm2 - PMinM2\\- 

However, it is important to note that acias{A,Tj^4^(^j\^^,Tj^^(^j^^) and a{A) are not 
necessarily the same. For example, take Ali — {(xi,X2, 0,0,X3) : x £ R'^} and 
M2 = {(xi,X2,x|,X3,X3) : X e M^}. Then Mi Ci M2 = {(x, 0,0, 0,0) : x e M} and 
is a regular point. Moreover Q:cias(0,T'_vii(o),rv(2(o)) = ^/^ whereas a(0) — 0. 
What goes wrong above is clearly that the two surfaces are tangential to each other 
in the direction (0, 0, 1, 0, 0), and it is intuitively clear that when this is not the case, 
the two concepts should coincide. To avoid such obstacles, we therefore introduce: 

Definition 4.3. A^i and M2 o,f£ said to be non-tangential at A if A is regular 
and they have a positive angle at A, i.e. if (7{A) < 1. We will often simply say that 
A is non-tangential. 

Note that the angle is always a number between and 7r/2, so in an intuitive 
sense only exceptional points do not satisfy non-tangentiality. We have no intention 
of formalizing this statement, but point out already that for the example considered 
in Section 2, this is indeed the case, which will be proven in [1]. 

Theorem 4.4. With the notation as in Section 2, we have that the set of tangential 
points between TZk and H is thin in "Hfc. 

In the general setting, pathological examples do exist. For example, take Mi ^ 
{(xi,X2,0) : X e K^} and M2 = {(xi,X2,x^) : x G R^}. However, when Mi and 
M2 are defined by polynomials, one can use similar methods, as in the proof of 
Theorem 4.4, to show that under mild conditions, if non-tangentiality holds at one 
point, then it holds everywhere except for a thin set. We will not pursue this. 

By (4.1) it is easy to see that A is non-tangential if and only if 

(4.3) {Tm, (A) e TM,nM2 (^)) n (t^, {A) Q TM,nM2 {A)) - {0} 
which in turn happens if and only if 

(4.4) TM^{A)f^TM2{A)=TM^nM2{A). 

This latter condition is usually referred to as clean intersection in microlocal anal- 
ysis. We have chosen the terminology non-tangential since it is more intuitive. We 
now show that non-tangentiality is a weaker concept than transversality, defined in 
(1.3). 

Proposition 4.5. Let Mi and M2 be two C^ -manifolds and A G Mi n M2 a 
transversal point. A transversal point A is also non-tangential. 



2/) 
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Proof. Applying the implicit function theorem to (j)i^4>2 easily yields that Mir\M2 
is a C^-manifold of dimension mi + ?7i2 — n aX, A. Thus Tmi (^) O TMir\M2 (^) has 
dimension n — 7712 and Ta42(A) TMir\M2{^) has dimension n — mi. Thus 

dim(rMi(A)erMinAl2(-4))+dim(TM2(A)erAlinA42(^))+dim(TMinA42(^)) = "•• 

But by transversality the sum of the above subspaces equals /C, which can only 
happen if (4.3) is satisfied. D 

We will denote M.i n AI2 by Al, and the objects from Section 3 associated to 
A^i, M.2 and M., e.g. </), by (/)i, (/)2 and respectively. We thus omit subindex when 
dealing with A^i n M.2- We now prove that for non-tangential points, the angle as 
defined here and the classical angle of the respective tangent spaces coincide. 

Proposition 4.6. If A is non-tangential, then 
Proof. Using (4.1) it is easy to see that 

^■(^) = \\PtmiPtm2 ^ Ptmihm^ II ^ \\PtmiPTm2 ~ PTMir\TM2 II = ^clas{A,TMiiTM2) 

where we used (4.4) in the crucial step. D 

Proposition 4.7. The function a in Definition 4-. 2 is C^^^ (on the set of regular 
points in Mi n ^2)- In particular, non-tangentiality is a local property, i.e. if A 
is non-tangential, then the same holds for all B € Mi D M.2 in a neighborhood of 
A. 

Proof. Let A e Mi C^M2 be a regular point and let 4> be the usual chart. We need 
to show that a o <}> is C^^^ around 0. By standard differential geometry (see e.g. 
[6]) there exists C^^ ^-functions /i, . . . , fn-m defined on the domain of such that 

{TMir\M2{(t>{x))) =Span {fi{x),. . . , fn~m{x)}. 

It is easy to see that (with j = \ 01 j = 2) 
(4.5) 
TMj{(f){x)) e TMinM2iHx)) = Span {PTA,/0(a:))/i(a;), • • • ^TA,/0(2:))/n-m(a;)}. 

Since the dimension of Tm {4'{x)) is constant mj , it is easy to see that the functions 
on the right are C^"^ as well. Moreover, as the dimension of the space on the left 
in (4.5) is constant mj — m, we can pick mj — m of the functions on the right and 
put them as columns of an injective matrix Mj (x) such that 

Tm.Wx)) QTM^nM2Wx)) = Ran M,{x) 
for X in a neighborhood of 0. By (4.1) we have for x around that 

(4.6) Cr((/)(a;)) = \\Pfi^„ Mi(x)PRan A'hixjW- 

(Note that this is not true with the classical definition when the two ranges have a 
non-trivial intersection, but with the definition here it works.) Moreover, it is easy 
to see that 

PRanM, ^ M,{M* M,)-^ M* , 

where x has been omitted for readability. Combining this with (4.6) and the fact 
that singular values are C°°-functions of the matrix entries, it is clear that a is 
C^^ near 0. as desired. D 
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5. Properties of the projection operators 

Let A All, A^2, (pi etc. be as before, i.e., C-manifolds with a non-tangential 
intersection point A. We assume that p > 1 and continue to use the convention of 
denoting objects related to the C^-manifold A^ = Ali n A^2 without subindex. In 
Propositions 3.2 and 3.3 the quantities S2 and S3 appears. These are not necessarily 
the same and they depend on an auxiliary constant 62 originating from Proposition 
3.2. In this section we let £2 be a fixed number (which we will determine later), 
and we let S4 denote the minimum of all possible s's from Section 3 related to the 
3 manifolds. The above will not be repeated in the statements of the results below. 
Thus, we can apply any result from Section 3 to either of the manifolds considered 
here. Moreover, letting j denote either 1, 2 or nothing, we let r4 > be such that 

Mj n B,c{A, S4) - (l)jiB{0, r4)) n Bic{A, 54), 

and such that the results of Section 3 applies to each 4>j{x) with |jx|| < r4. We also 
assume that £2 < 1- 




Ml nM2 



Figure 6. Illustration of the difference between pi and tti. V and 
£ appear in the proof of Proposition 5.2. 



Given an afline linear manifold J\f d IC, we denote by Pj^ the orthogonal projec- 
tion onto A/". We introduce maps pj : B{A, S4) -^ JC, ( j = 1 or j = 2), via 



Pj{B) 



BfM,(^iB))iBy 



Thus, Pj resemble ttj but is slightly different, ttj projects onto Aij whereas pj 
projects onto the tangent plane of A^^ taken at the closest point to B in AlinAl2! 
i.e. 7r(i?), (see Fig 5). A proper estimate of the difference is given in Proposition 
5.2. 

Lemma 5.1. The operators pi and p2 are C^^^-maps in Bic{A,S4). Moreover, we 
can select a number s 5 < 84 such that the image ofB{A,sz) under pi, p2,T^.,'n:i, 1^2, 
as well as any composition 0/ two of those maps, is contained in B{A, S4). 

Proof. The second part is an immediate consequence of the continuity of the maps. 
Let j denote 1 or 2 and set Mj{B) = d(j)j{Tr{B)). Note that Mj is a C^^^-map in 
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Bic{A, S4) by Proposition 3.3 and the choice of 34. Moreover, 

P^B) = Pf^^(,(B))(S) - AB) + Pt^^mb)){B - 7r{B)) = 
= 7t{B) + M,(B){M*{B)M,(B))-'m*{B){B - tt{B)), 
from which the result fohows. D 

Proposition 5.2. Given any B G B{A, S5) and j — I or j — 2, we have 

\\nj{B)-p,{B)\\<5^\\B-n{B)\\. 

Proof. By Lemma 5.1 we have that Proposition 3.2 apphes to the point C = 7r(_B). 
It is no restriction to assume that tt{B) — 0, which we now do. Denote V ~ Tj^ . (0), 
£ ^ Span {B - Pj{B)} and J" == /C e (X* £), (see Fig. 5). Let Db and Eb be 
elements of V and £ such that B = Db + Eb , and note that 

p,iB) = DB. 
We thus have to show that 

(5.1) \\n,{B)-DB\\<5^\\B\\. 

First note that by Proposition 3.2 (ii) there exists a point mAiir\Ai2<^l3(DB, <^2\\Db\ 
Thus 

\\B - TTj{B)\\ <\\B-Db\\+ eapsll - ll^sll + e2\\DB\\ 

and hence ttj{B) is a member of the set 

(5.2) [{D,E,F) : \\D - Db\\^ + \\E - Eb\\^ + \\Ff < (||Sb|| + eapsID^}. 
However, by Proposition 3.2 (i) we have that ttj{B) also is a member of 

(5.3) {iD,E,F):\\E\\' + \\Fr<el\\D\\'}. 

The left hand side of (5.1) is thus dominated by the supremum of the function 

f{D,E,F) = \\D-DBf + \\E\\^ + \\Ff, 

subject to the conditions in (5.2) and (5.3). Either by geometrical considerations 
or the method of Lagrange multipliers, it is not hard to deduce that this supremum 
is attained for F = and D,E oi the form D = (IDb and E = cEb where d, e e M. 
We now have a two-dimensional problem of circles and cones, and in the remainder 
of the proof we treat D and E as elements of M? . Given D + E vn the intersection 
of (5.2) and (5.3), it is easily seen (see Fig. 5) that 

ll^ll <e2{\\DB\\ + \\EB\\+e2\\DB\\), 

since this is the height of the cone at the outer edge of the circle. The problem 
becomes simpler if we replace the cone in (5.3) by the following strip: 

(5.4) {S : ll^ll < e2((l + €2)\\Db\\ + \\Eb\\)]. 

From Figure 5, and some freshman formulas, it is readily verified that the sought 
supremum is dominated by Q;(||i?B|| + e2||£'B||), where a is the angle given by 

_^(\\EB\\-e2{{l + €2)\\DB\\ + \\EB\ 

a = cos ' 



\EB\\+e2\\DB\ 
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e2\\Di 



e2{{l + e2)\\DB\\ + \\EB\\) 



Figure 7. Proof illustration. 



Since we have assumed that 62 < 1, the proposition thus follows if we establish that 

' \EB\\-€2mDB\\ + \\EB\\)\ \\EB\\+e2\\DB\ 



COS 



< ^v^ 



(II^bII +£211^511) J .JW^w^ + wdbV 

By dividing with ||-E_b|| at suitable places, one sees that this is equivalent to 

'l-2e2t-e2\ 1 + 620 . /— 



sup -^ cos 

t>0 



1+62* J vm^ 

Using cos^^(l — x) < TT^Jxpl and £2 < 1 we obtain 



cos 



l-2e2t-e2\ 1 



1 + 62* 



< 



TT /3e2i + e2 1 



V2 



VT 

£2^ 



£2^ -1 

= COS 



1- 



3e2i + e2\ 1 + 62* 



1 + £2* / VT+f2 



f2 



t: V(3t+l)(l + t) ^^ 
< V£2^=-^^ , „ < 5Ve2 



< 



1 + £2* \/r+t2 - ^ V2 vr+i2 

since the function on the right clearly is bounded, and a few calculations show 
that the maximum is attained at t = (1 + v5)/2 which gives 5-^62 as an upper 
bound. D 

Lemma 5.3. Given B G B(A, S5) we have 

7T{B)=n{p,(B))=7r{p2{pi(B))). 

Proof. If we prove the first equality the second follows by reversing the roles of 1 
and 2 and applying the first equality to pi{B). To see the first equality, note that 
B - pi{B) _L TM^{^T{B)) and obviously B - tt{B) _L TM^nAhi^B)) so 

Pi{B) - 7r{B) = (piiB) -B) + {B- ti{B)) ± Tm^hmA^B)), 

which by Lemma 5.1 and Proposition 3.3 implies ■n{B) = Tr{pi{B)), as desired. D 
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Lemma 5.4. Let C5 > 1 and 65 > be given. If E,F £ K. satisfies \\E\\ > C5||F| 

and \\E — F\\ < €5, then 

\\E\\<e,^. 
C5 - 1 

Proof. If II £^11 < £5 we are done, otherwise 

\\E\\ \\E\\ 



l^ll \\E\\-e5 
which easily gives the desired estimate. D 

The next result will be the main tool for proving convergence of the alternating 
projections. 

Theorem 5.5. Let AGAlinA^2 be a non-tangential point and assume that Mi 
and AI2 are C^ -manifolds. Then for each cg > a {A) and each eg > there exists 
a positive sg < S5 such that for all B G M2 H B{A, sg) we have 

(i) \\Tri{B) - 7r{B)\\ < ce\\B - 7r{B)\\ 
(it) \\tt{tti{B)) - n{B)\\ <ee\\B-n{B)\\ 

Moreover the same holds true with the roles of A4i and M2 reversed. 
Proof. Fix ci such that (t{A) < ci < cg and pick an 57 < S5 such that 

(5.5) sup{ct(C) : C g A^i n AI2 n B{A, 57)} < ci, 

which we can do since a is continuous by Proposition 4.7. Let C2 > 1 be such 
that C2C1 < Cg. By Lemma 5.1 and Proposition 3.3, pi and it are C^-functions, and 
hence we can pick C3 > such that 

(5.6) ||p,(i?) - p,{B')\\ < c,\\B - B'W and ||7r(i?) - 7r(S')|l < c^WB - B'\\ 

for all 5, B' G B{A, 54), (recall that S4 was chosen in the beginning of this section). 
Finally, we fix £2 such that 

(5.7) 5Ve2(c3 + C3) < eg and 5^/e^(l + C3) < cg and (1 + 5yeJ)c2Ci < cg. 

C2 — t 

This may seem like a circle argument, because ci,C2 and C3 depends on cg which 
depends on e2 via S5, and the c^'s appears in (5.7). However, this is easily circum- 
vented by first choosing S5 with e2 = 1, say, and pick values of ci, C2, C3. Then, 
once the real e2 has been chosen via (5.7) we can redefine 54, S5 and 57 accordingly 
without violating (5.5) or (5.6). 
Now, pick an sg such that 

Ti{B{A,s^))(zB{A,sr), 

and let B G M.2 H B{A^S(f). We begin with proving [ii). Denote C ~ 7r(_B) and 
D = -Ki{B) and note that C G Mi C\ M2 n B{A,sj) so ct(C) < cg. There is no 
restriction to assume that C = 0, which we do from now on. We thus need to show 
that \\tt{D)\\ < eg||B||. Put B' = p2{B) and D' = pi{B'). (See Figure 5 and recaU 
the tt{B') — tt{B) = by Lemma 5.3). First, note that by Proposition 5.2 

(5.8) \\B - B'W = ||7r2(B) - P2{B)\\ < 5^\\B\\ 
and moreover by (5.6) and Proposition 5.2 we have that 

11^ - ^'11 = \\Pi{B') - ni{B)\\ < \\pi{B') pi{B)\\ + WpiiB) 7ri(B)|| < 
^ ' ' < C3\\B' - B\\ + 5V^\\B\\ < 5Vi;(l + C3)||S|| 
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By Lemma 5.3 we have — tt{B) — Tr{D') so part (m) follows by (5.6), (5.7), (5.9) 
and the calculation 

||7r(i^)|| = \\n{D) 7t{D')\\ < cs{5^,{l + c,)\\B\\) < e,\\B\\. 

We turn to part (i). Clearly B' E 7^42 (0) ^^"^ by Lemma 5.3 we also have D' E 
r^,(0).Thus 

(5.10) 



1^ 

IS'II 



< ct(0) < ci 



whereas (i) amounts to showing that ||£)||/||i?|j < cq. Recall (5.9) and apply Lemma 
5.4 with E = D, F = D' , £5 = 5^/e^(l + C3)|lB|| and C5 = 02- We see that either 

lli^ll 



(5.11) 



\D'\ 



<C2 



C3, 



C2-1 



LB 1 1 . in which case we are done since the constant is 



or \\D\\ < 5Ve^(l 

less than cg by (5.7). We thus assume that (5.11) holds. Note that 

\\B'\ 



(5.12) 



\B\\ 



<1+5V^ 



by (5.8). Combining (5.10), (5.11) and (5.12) we get 

lli^ll \\D\\ WB'WWD' 



\B\\ 



\D'\\ \\B\\ \\B' 



< C2(l + 5Ve^)ci < cg 



by (5.7). 



D 



6. Alternating projections 



We are finally ready for the main theorem. The third conclusion below was not 
mentioned in the introduction. It basically says that if the angle between Ai 1 and 
7W2 is not too close to 0, the sequence of alternating projections Bq, Bi, B2, ■ ■ ■ 
will converge within machine precision within a fairly low number of iterations. In 
the terminology of [8], (-Bfe)^o converges "R- linearly" with rate less than c. For 
example, if (j{A) — 1/2, then we will hit single precision (« lO"'''^) at B24, since 
single precision has 24 bits of significant precision. 
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Theorem 6.1. Let Jvli n M2 be locally non-tangential C^ -manifolds around A G 
All n A^2, and let e > and 1 > c > cr(A) be given. Then there exists an s > 
such that the sequence of alternating projections 

Bo=^i(S), Bi-^2(7ri(B)), B2=7ri (^2(^1(5))), B3 - 7r2(^i(7r2(7ri (B) ))),.. . 



(i) converges to a point Boo G All H AI2 

(ii) \\B^-7r{B)\\ <e||B-7r(B)|| 

^li) \\Boo-Bk\\ <c'=||B-^(B)|| 
Proof. Let cq and eg in Theorem 5.5 be given hy cq — c and 
(6.1) ee = {l-c)e/2. 

Let sg be given by Theorem 5.5 and pick 

S6{l-e) 



(6.2) s < 



4(2 + e) 



such that tt{B{A,s)) C B{A,se/4:), (recah that e < £2 < 1; by assumption). The 
latter condition ensures that 

(6.3) ||7r(B)-A|| <S6/4. 
Let I = \\B — 7r(_B)|| and note that 

(6.4) 1<\\B-A\\ + \\A - 7r{B)\\ <s + sq/A. 

First note that \\Bo-B\\ = ||7ri(B)-B|| < ||7r(B)-B|| ^l and that 7r(B) = tt{Bo) 
by Lemma 5.3, so 

\\Bo - 7:{Bo)\\ <\\Bo- B\\ + \\B- tt{B)\\ < 21. 

Applying Theorem 5.5 we get 

\\Bk+l - TT{Bk+l)\\ < \\Bk+l - TT{Bk)\\ < c\\Bk - TT{Bk)l 

as long as 

(6.5) BkeB{A,se). 
Assuming this for the moment we get 

(6.6) \\Bk ~ TTiBk)\\ < 21c'' 
and (Theorem 5.5 and Lemma 5.3) 

(6.7) MBk+i) - 7:{Bk)\\ <ee{2lc''). 

The sequence {TT{Bk))'^i is thus a Cauchy sequence, and hence converges to some 
point Boo- By (6.6) the sequence (-Bfc)^i must also converge, and the limit point is 
again B^o, which thus satisfies B^o = 7r(-Boo) since tt is continuous. By the triangle 
inequality, the fact Tr{B) = 7r(i3o), (6.1) and (6.7) we have 

\\n{Bk)^7T{B)\\<pL<el, 
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and combining this with (6.3), (6.4), (6.6) we also have 

\\A - BkW <\\A- n{B)\\ + ||7r(B) - n{Bk)\\ + MBk) - B^W < s^/A + el + 2l< 
< se/4 + e(s + se/4:) + 2{s + se/4) < sg, 

where the last inequality follows by (6.2). With these estimates at hand, it is easy 
to turn the above argument into a proper induction proof in which (6.5) is verified 
at each step. We omit the details. D 

7. Rank k matrices versus Hankel matrices; numerical examples 

We now continue the example in Section 2 and the application to approximation 
of a signal by sums of k exponential functions, as outlined towards the end. By 
Theorem 4.4, an arbitrary intersection point of TZk and Ji is almost surely non- 
tangential. Given the signal / we let g be the closest "sum of k exponentials" to / 
in the || • ||^-norm, see (2.6). Theorem 6.1 thus says that if ||/ — .gllwi is not too large, 
the sequence of alternating projections will converge to an H{goo), where g^c ~ g 
is a sum of k exponentials. More precisely, let 

Then for every e > there exists an s > such that if dist(/, JC) < s, then 

||.goo - g\\w < e dist(/, /C). 

We now discuss what happens if / is not close enough to g that Theorem 6.1 
can be applied for any e. Since both n-u and tt-r,^ are contractions, the sequence 
of alternating projections (Bj)°^q (with Bq = H{f)) will be bounded. Thus it has 
a convergent subsequence, and the limit point is easily seen to be in "Hfe. It is not 
hard to deduce that dist(i?j, "H^) — > as j — >■ oo. The only way the whole sequence 
could avoid converging is thus if it switches endlessly along the valleys of the thin 
set of tangential points, thereby avoiding the surrounding hills made by the open 
sets UAe«fc'8(v4, s^i), where s^i > is given by Theorem 6.1 (with e = 1 say) for all 
non-tangential yl's, and s^i = for the thin set of tangential A's. That this could 
happen seems highly unlikely to us, and we have certainly never encountered it in 
practice. However, we leave it as an open problem to prove that this can not occur. 

For practical purposes, it is of course of interest that s be a relatively large 
number, (given some fixed e and c). Inspecting the proof Theorem 6.1, one gets 
the impression that B{A, s) will not be distinguishable even with binoculars. To 
test the actual relationship between s and e in Theorem 6.1, (with c = 1), we 
conducted a few experiments which we now present. A function^ g is written as 
a sum of 10 exponential functions, and then / is generated by adding a smaller 
function n to g. g is normalized such that \\g\\w ~ 1, (recall (2.6)), n is chosen 
such that H{n) is orthogonal to Hiq at Hg, (with respect to the Hilbert-Schmidt 
norm). By Proposition 3.3, H{g) is likely very close to 7r(iJ(/)), and so we will 
estimate d\st{H{f),Hio) = \\H{f) - Tr{H{f))\\ by \\H{f) - H{g)\\ = ||n||^ and 
||-H^(/oo) — T^(H{f))\\ by \\H{fao) — H(g)\\. Note that there is no explicit way to 



In this section we will sloppily say function when we talk of its sampling, i.e. a sequence. 
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compute 7r(-ff(/)). We set 

e{g,n) = 



\\Hif^) - H{g)\\ _ \\.foo-g\\ 



\H{n)\ 



Let s be a parameter taking values 0.1,0.01, 0.001, 0.0001 and 0.00001. For each 
value of s we randomly generate 50 normalized g^s as above, and for each g we 
randomly generate a n with ||n||^ = s. The supremum of s{g,n) should give us 
an idea of what value of e allows for such an s as in Theorem 6.1. The results 
are demonstrated in Figure 8 and Figure 9. The difference between the two is that 
Figure 8 used white noise, and in Figure 9, the "noise" consisted of sums of randomly 
generated exponentials. Judging from these figures, e ~ a seems to be a good rule 
of thumb, although this will of course vary substantially from one application to 
another. The method seems to work slightly better for the case of white noise. 







10 20 30 40 50 60 70 80 90 100 



Figure 8. Signal with 10 exponentials and white noise. 



8. Summary and open problems 

We have developed a theoretical framework that enables us to understand con- 
vergence properties of alternating projection schemes. This provides substantially 
stronger results than previously available, e.g., for theorems relying solely on Zang- 
will's Global Convergence Theorem, which neither provide convergence rate esti- 
mates nor information about how far away the computed approximation is from 
the optimal value. Moreover, in contrast to the theory developed for the case of 
transversal manifolds, our framework provides more information under very non- 
restrictive requirements on the manifolds. 
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Figure 9. Signal with 10 exponentials and sums of other expo- 
nentials added. 

We end by noting a few open problems. We first discuss smoothness. By ex- 
amples similar to Example 1.2, it is not hard to see that one needs the manifolds 
to be at least C^ for the alternating projections to converge. By inspection of the 
proofs, one easily sees that for Theorem 6.1 to hold, it suffices with C}^^ , i.e. that 
d(j) is Lipschitz continuous. It is thus an open question whether Theorem 6.1 is 
true assuming only C^. Next we discuss non-tangentiality. Again, Example 1.2 
can be tailored such that the alternating projections does not converge, despite 
assuming C°°, if we allow the manifolds to be tangential. However, for all appli- 
cations we are aware of, the manifolds are algebraic, and it seems quite unlikely 
that a similar thing could happen in this case. We conjecture that for algebraic 
manifolds, the sequence of alternating projections will always converge, without any 
additional assumptions. Recall Example 1.4, where the sequence clearly converges 
albeit extremely slowly. For practical purposes, non-tangentiality is thus still vital. 
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